Searching the Físchlár-NEWS Archive on a Mobile Device

The Físchlár-NEWS system provides web-based access to an archive of digitally recorded TV News broadcasts over several months, and has been operational for over a year. Users can browse keyframes, search teletext and have streamed video playback of segments of news broadcasts to their desktops. This paper reports on the development of mFíschlár-NEWS, a version of Físchlár-NEWS which operates on a mobile PDA over a wireless LAN connection. In the design and development of mFíschlár-NEWS we have realised that mobile access to a digital library of video materials is more than just the desktop system on a smaller screen, and the functionality and role that information retrieval techniques play in the mFíschlár-NEWS system are very different to what is present in the desktop system. The paper describes the design, interface, functionality and operational status of this mobile access to a video library. 1. The Físchlár NEWS Archive The Físchlár-NEWS Archive is an online repository of daily evening TV news which is broadcast by the Irish National broadcast channel, RTE1. The system automatically records and processes news programmes along with their teletext (close-captioned subtitle of what is said during the programme) every evening and allows web-based access to the indexed programmes by both browsing and searching (querying). Searching is an important access mechanism for any archive of considerable size, and this is also true in the Físchlár-NEWS Archive. All news programmes in the archive are accessible by browsing the reverse-chronologically ordered list of programmes, and thus the latest news programmes (such as yesterday evening’s news) can be more easily viewed. However, immediate access to segments of the news which a user may want to see by allowing querying of the entire archive is a powerful alternative and complimentary access to browsing the archive. The Físchlár-NEWS Archive uses the teletext captured along with the video signal, for subsequent indexing and retrieval. Figure 1 shows a screen shot of searching through the Archive. A user types in query term(s) into the query box at the top-left side of the screen, and the result of the matching is displayed below, Figure 1: Físchlár NEWS Archive – the full desktop system interface where matching news segments are grouped by their entire news programmes and each of the retrieved segments within a news programme show their time of occurrence within that programme. When the user clicks on one of the matched segments, the programme will open on the right side of the screen in a content browser, with the matched point within the programme displayed. The browser shows keyframes and teletext together and the user can choose to further browse other parts of the programme or start playback at any point in the programme by simply clicking on a keyframe. The Físchlár-NEWS Archive is an adapted version of the Físchlár Digital Video System which records any TV broadcast programmes on users’ requests and allows browsing/playback within the University’s computer laboratories and campus residences from a desktop computer environment [O’Connor et al. 00]. The Físchlár system has been in operation on the University campus for almost 3 years and is accessed via a web browser on a desktop computer. At the time of writing we have over 1,700 registered users, of whom about half are “active” and regular users. Físchlár is used for research, for teaching and learning, and for entertainment. The Físchlár-NEWS system has been in operation for about a year and was developed specifically for students of journalism and media studies to provide them with easy access to an archive of broadcast TV news. Físchlár (which allows users to record any broadcast TV program) and Físchlár-NEWS (which records only the main evening news) are both very popular among the University community. As an indicator of what it offers, the full Físchlár-NEWS Archive provides the following features for users: 1. Browsing of the list of daily news programmes in reverse-chronological order (by scrolling); 2. Text-based searching of all the news programmes by inputting word(s), with the resulting matched segments displayed by highlighting the matched word(s) with 3-4 lines of surrounding text providing the context of the matching; 3. Browsing individual news programmes with one of our various content browsers (keyframebased browsers; for more detail see [Lee & Smeaton 02]); 4. Playback of news programmes by clicking on any of the keyframes in a content browser. Our work in the development of Físchlár for a desktop environment has always been based on user issues and considerations – designing, building and deploying an operational Físchlár system whose interface functionality and layout has been designed with the user’s goals, tasks and abilities in mind. This approach of leading the development of an interface to a video library system from the user’s perspective, is continued in the development of a mobile handheld platform interface to FíschlárNEWS. The most important set of issues which we will address in the development of FíschlárNEWS are those of interface design – general interface concerns for mobile applications as covered in section 2 and interface design concerns for accessing our system in section 3. In section 4 we present the design and implementation of Físchlár-NEWS for a Compaq iPAQ, a PDA connected to the Físchlár database via a wireless LAN connection, and in section 5 we present our conclusions. 2. Interface Design Considerations for IR from Mobile Devices User interface design for mobile devices has been discussed for many years now, with the consensus that a mobile interface should require a different interaction style from that of the GUI desktop interface, and that attempts to cram all the functionality of desktop system into a mobile device are a mistake [Thomas et al. 95] [Marcus et al. 98] [Kristoffersen & Ljungberg 99] [Longoria 01]. Though the current literature alerts to the fact that we do not have any established or known methodology on which to base an interface design for a mobile platform, rough design guidelines have been suggested based on experiences of individual researchers. These are: • Determine what kind of information and what style of interaction a user will need/want to access via the mobile device, in order to have most benefit [Ruuska-Kalliokulju et al. 01] [Jordan et al. 01] [Longoria 01]; • Minimise user input – where applicable, provide simple user selections such as yes/no options, simple hyperlinking by tapping, etc. instead of asking the user to articulate query formulation or use visually demanding browsing that requires careful inspection of the screen; • Filter out information so that only a small amount of the most important information can be quickly and readily accessed via the mobile device (e.g. use of automatic recommendation as provided in the Físchlár TV system [O’Sullivan et al. 02]); • Proactively search and collect potentially useful pieces of information for a user and point these out, rather than trying to provide full coverage of all information via an elaborate searching/browsing interface; • Use a layout that does not require a large space – converting where applicable, spatial information into a temporal format (as in RSVP [Bruijn & Spence 00]); In terms of developing any system for a mobile device which is to support searching and information retrieval tasks, all these guidelines point to more pre-processing on the system’s side in order to determine which pieces of information a particular user will most likely to want to see. This encourages the development of systems which proactively recommend a particular piece of information (or pointers) to the user, and consequently demand less interaction on the user’s part. This aspect is even more important in the case of information retrieval from a video archive where browsing is such an important component of video access. What all this means is that in the development of search systems to be accessed from mobile platforms, the information retrieval functionality should be hidden as much from the user as is possible, and should form part of the information pre-processing. We shall see more of this in the next section. 3. Interface Design Concerns for Accessing the mFíschlár-NEWS Archive In this paper we are concerned with developing a version of Físchlár-NEWS which is accessed from a mobile handheld platform. Our target user would use “mFíschlár-NEWS” from a mobile PDA such as a Compaq iPAQ, connected to the network with enough bandwidth to support streaming video. Before we delve into the way in which we have designed and built mFíschlár-NEWS, it is worth examining what are the characteristics of the TV news archive we are providing access to. The RTE1 main evening news is a daily broadcast which summarises news developments over the previous 24 hours into a single 30-minute programme, with an advertisement break in the middle. It is typical of most TV news programmes in that it begins with a snapshot of the headlines followed by the signature tune and signature introduction and then a series of about 8 or 9 individual news stories before finishing up with a weather forecast. Each of the individual news stories is composed of a series of shots of an anchorperson with an illustrative graphic in the background (see the playback window in Figure 1 where the frame is taken from a story about a “Mother’s Appeal”), interspersed with outside footage. Within a single day’s news, some of the stories may be new, reflecting a new story which may unfold over time (first reporting of joyriders crashing into a police car) or may begin and end on that day (farmers protest at rising milk prices), some may be updates on a continuously developing story (Israeli siege of church in Bethlehem) or some may be updates on an old story from the past (killer sentenced to prison, reference to previous stories on murder, capture and trial). As an indication of the structure of the RTE1 news, we took a period of 28 days during November 2001 and we analysed this to determine the characteristics of breaking news items during this period. Appendix A presents a graphical summary of how the stories in each broadcast are related to each other. Each column in that appendix represents a date, each row represents a unique news story and shaded boxes across the row represent occurrences of that news story on that date. A series of shaded boxes across a row represents repeated occurrences of the same story across different days. There is of course an element of subjectivity in the way in which we have manually determined the relationship between stories and whether a news story on a given day is a new story or an update on an older one. Nevertheless we believe the characteristics we have identified are similar to those used in datasets from recent work on Topic Detection and Tracking [Allan et al. 98] Our analysis of this period of news has identified 231 stories (about 8.25 per day) of which 115 are unique. On a given day an average of 2.89 of the stories presented on the RTÉ1 news are updates or developments on continuing stories from the previous day, 1.46 are updates on stories from somewhere prior to that, and 3.89 are new stories. The constantly appearing story across the 28 dates in Appendix A is about the Afghanistan conflict and some of the once-off stories are about things like the Government announcing a new health strategy and a US company cloning the first ever embryo. The analysis of stories in the RTÉ1 news programme was important in helping us to design the functionality of a mobile platform access to this archive. The kind of work tasks which a mobile platform will naturally support are somewhat different to those tasks supported from a desktop environment. Because of screen size, absence of a keyboard and mouse, awkward data entry facilities, limited pointing ability and the fact that mobile users cannot be relied upon to have desk space around them, mobile user access to a video archive will generally support rather simple information seeking tasks. Sophisticated information seeking such as trying to find obscure TV News references to persons which may or may not even be in the archive, are best done from a desktop environment. Drawing up a simple usage scenario for a mobile user accessing a video archive may help [Carroll 00] in defining what a mobile device for accessing the Físchlár-NEWS Archive, is best suited for: A user sitting in a bus (or waiting for a bus on a bus stop) brings out her PDA from the pocket to get recent updates of news stories since she last accessed the archive, either from her desk or PDA. The above scenario, though rough and simple, implies a few important information retrieval techniques which are required as part of the system side processing: • Video shot boundary detection and keyframe selection to allow keyframe browsing of a news story; • Story identification and segmentation within a broadcast TV news programme; • Linking of related stories in the news archive; • Summarisation of a news story both visually (selection of keyframe) and as text (selection of teletext fragments); • Identification of an individual user and tracking his/her access activity (through a user profile); • Determination of which news stories on which date(s) would be most interesting or useful for a particular user at the moment of access; While the kind of information retrieval techniques needed to support the above are sophisticated, on the other hand, the scenario implies quite a simple user interface, places little demand on the user, and has its main emphasis on good summarisation of each recommended story. From the interface perspective, it requires • A listing of new and updated news stories in summarised format; • An option for further, more detailed browsing of a story and for playback of that story; • A temporal keyframe browser [Lee et al. 01]; As most work will have gone to the indexing-time processing of the video, and the resultant news listing will show only new or updated news stories, all a user should have to do is to tap on a title from one of the latest stories, and within a list of news items within a story, simply tap on an item to play or browse. This saves the user from elaborate and visually-demanding interaction from – which is mostly required on a desktop applications – and makes the interaction a more casual tapping action which does not require constant and uninterrupted attention, which is suitable for a mobile environment. This is a different philosophy from our desktop-based interface design for accessing Físchlár-NEWS as shown earlier in section 1, which allows full access to the news archive with a chronological listing of all news programs, keyword searching, and various keyframe browsers with elaborate interaction of the user’s choice. The user accessing via desktop in an office will want not only recent updates to the TV news archive but may also want to study and research from within the news archive, and thus will benefit from such elaborate provision of features. Figure 2 below illustrates the differences between the features provided on the desktop and on the mobile device for access to the Físchlár News Archive: Note that Playback and a keyframe-based browser on the desktop environment have been determined as useful for a mobile environment, whereas the chronological index, keyword searching and various spatial browsers have been decided unsuitable for mobile environment.